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THE EXPECTED NORM OF A SUM OF INDEPENDENT RANDOM MATRICES: 
AN ELEMENTARY APPROACH 

JOEL A. TROPP 


Abstract. In contemporary applied and computational mathematics, a frequent challenge is to bound the 
expectation of the spectral norm of a sum of independent random matrices. This quantity is controlled by 
the norm of the expected square of the random matrix and the expectation of the maximum squared norm 
achieved by one of the summands; there is also a weak dependence on the dimension of the random matrix. 
The purpose of this paper is to give a complete, elementary proof of this important, but underappreciated, 
inequality. 


1. Motivation 

Over the last decade, random matrices have become ubiquitous in applied and computational math¬ 
ematics. As this trend accelerates, more and more researchers must confront random matrices as part of 
their work. Classical random matrix theory can be difficult to use, and it is often silent about the ques¬ 
tions that come up in modern applications. As a consequence, it has become imperative to develop and 
disseminate new tools that are easy to use and that apply to a wide range of random matrices. 

1.1. Matrix Concentration Inequalities. Matrix concentration inequalities are among the most popu¬ 
lar of these new methods. For a random matrix Z with appropriate structure, these results use simple 
parameters associated with the random matrix to provide bounds of the form 

E||Z-EZ|| < ... and P{||Z-EZ|| > t] < ... 

where || • || denotes the spectral norm, also known as the £2 operator norm. These tools have already 
found a place in a huge number of mathematical research fields, including 


numerical linear algebra [Troll] 
numerical analysis [MB14] 
uncertainty quantification [CG14] 
statistics [Kolll] 
econometrics [CC13] 
approximation theory [CDL13] 
sampling theory [BG13] 
machine learning [DKC13, LPSS + 14] 


learning theory [FSV12, MKR12] 
mathematical signal processing [CBSW14] 
optimization [CSW12] 
computer graphics and vision [CGH14] 
quantum information theory [Holl2] 
theory of algorithms [HO 14, CKM + 14] and 
combinatorics [OlilO] . 


These references are chosen more or less at random from a long menu of possibilities. See the mono¬ 
graph [Trol5a] for an overview of the main results on matrix concentration, many detailed applications, 
and additional background references. 
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1.2. The Expected Norm. The purpose of this paper is to provide a complete proof of the following im¬ 
portant, but underappreciated, theorem. This result is adapted from [CGT12, Thm. A.l], 

Theorem I (The Expected Norm of an Independent Sum of Random Matrices). Consider an independent 
family {Si,..., S n } of random d\ x r/ 2 complex-valued matrices with ES,- = 0 for each index i, and define 

z-=L'LSi- a.l) 

Introduce the matrix variance parameter 

i/(Z):=max{||E[ZZ*]||, ||E[Z*Z]||} 

= max{E | .E[S i s;]||,E i E[S I TS i ]||} (L2) 

and the large deviation parameter 

I:=(Emax ! -||S ! -|| 2 ) 1/2 . (1.3) 

Define the dimensional constant 

C{d) := Cidijdf) := 4• (l + 2[log(<ii + c? 2 )l)- (1-4) 

Then we have the matching estimates 

y/c-vlZ) + c-L < (E||Z|| 2 ) 1/2 < \/C{d)-v{Z) + C{d)-L. (1.5) 

In the lower inequality, we can take c := 1/4. The symbol ||-|| denotes the (. 2 operator norm, also known 
as the spectral norm, and * refers to the conjugate transpose operation. The map [•] returns the smallest 
integer that exceeds its argument. 

The proof of this result occupies the bulk of this paper. Most of the page count is attributed to a detailed 
presentation of the required background material from linear algebra and probability. We have based 
the argument on the most elementary considerations possible, and we have tried to make the work self- 
contained. Once the reader has digested these ideas, the related—but more sophisticated —approach in 
the paper [MJC + 14] should be accessible. 

1.3. Discussion. Before we continue, some remarks about Theorem I are in order. First, although it may 
seem restrictive to focus on independent sums, as in (1.1), this model captures an enormous number of 
useful examples. See the monograph [Trol5a] for justification. 

We have chosen the term variance parameter because the quantity (1.2) is a direct generalization of 
the variance of a scalar random variable. The passage from the first formula to the second formula in (1.2) 
is an immediate consequence of the assumption that the summands S* are independent and have zero 
mean (see Section 5). We use the term large-deviation parameter because the quantity (1.3) reflects the 
part of the expected norm of the random matrix that is attributable to one of the summands taking an 
unusually large value. In practice, both parameters are easy to compute using matrix arithmetic and 
some basic probabilistic considerations. 

In applications, it is common that we need high-probability bounds on the norm of a random ma¬ 
trix. Typically, the bigger challenge is to estimate the expectation of the norm, which is what Theorem I 
achieves. Once we have a bound for the expectation, we can use scalar concentration inequalities, such 
as [BLM13, Thm. 6.10], to obtain high-probability bounds on the deviation between the norm and its 
mean value. 

We have stated Theorem I as a bound on the second moment of ||Z|| because this is the most natural 
form of the result. Equivalent bounds hold for the first moment: 

\Jc'-v{Z) + c'-L < E || Z|| < \/C(.d)-v[Z) + C(d)-L. 

We can take c' = 1/8. The upper bound follows easily from (1.5) and Jensen’s inequality. The lower bound 
requires the Khintchine-Kahane inequality [L094] . 
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Observe that the lower and upper estimates in (1.5) differ only by the factor C{d). As a consequence, 
the lower bound has no explicit dimensional dependence, while the upper bound has only a weak depen¬ 
dence on the dimension. Under the assumptions of the theorem, it is not possible to make substantial 
improvements to either the lower bound or the upper bound. Section 7 provides examples that support 
this claim. 

In the theory of matrix concentration, one of the major challenges is to understand what properties 
of the random matrix Z allow us to remove the dimensional factor C(d) from the estimate (1.5). This 
question is largely open, but the recent papers [01il3, BH14, Trol5b] make some progress. 

1.4. The Uncentered Case. Although Theorem I concerns a centered random matrix, it can also be used 
to study a general random matrix. The following result is an immediate corollary of Theorem I. 

Theorem II. Consider an independent family {Si,..., S,J of random d\ x d 2 complex-valued matrices, not 
necessarily centered. Define 

*--=LUst 

Introduce the variance parameter 

v{R) :=max{||E[(H —E/I)(« —EU)*]||, ||E [{R - Eft)* (R - El?)] ||} 

= max{ | £” = ! E [ ISi - E S,0 (S f - ES f )* ] ||, || E [ (S,- - E S,0 * (S/ - E S,0] ||} 
and the large-deviation parameter 

L 2 := EmaX; ||S; - ES,-|| 2 . 

Then we have the matching estimates 

\Jc- v{R) + c-L < (E||ft-Eft|| 2 ) 1/2 < y/C(d)-v{R) + C{d)-L. 

We can take c = 1/4, and the dimensional constant C[d) is defined in (1.4). 

Theorem II can also be used to study \\R\\ by combining it with the estimates 

||Eft|| - (E||J?-EJ?|| 2 ) 1/2 < (E||j?ii 2 ) 1/2 < ||E|| + (E||ft-Eft|| 2 ) 1/2 . 

These bounds follow from the triangle inequality for the spectral norm. 

It is productive to interpret Theorem II as a perturbation result because it describes how far the ran¬ 
dom matrix R deviates from its mean Eft. We can derive many useful consequences from a bound of the 
form 

(Elift -Eft || 2 ) 172 < ... 

This estimate shows that, on average, all of the singular values of ft are close to the corresponding singu¬ 
lar values of Eft. It also implies that, on average, the singular vectors of ft are close to the corresponding 
singular vectors of Eft, provided that the associated singular values are isolated. Furthermore, we dis¬ 
cover that, on average, each linear functional tr[Cft] is uniformly close to Etr[Cft] for each fixed matrix 
C e M rfzXdl with bounded Schatten 1-norm Hdlsj < 1. 

1.5. History. Theorem I is not new. A somewhat weaker version of the upper bound appeared in Rudel- 
son’s work [Rud99, Thm. 1]; see also [RV07, Thm. 3.1] and [Tro08, Sec. 9]. The first explicit statement 
of the upper bound appeared in [CGT12, Thm. A.l], All of these results depend on the noncommuta- 
tive Khintchine inequality [LP86, Pis98, BucOl]. In our approach, the main innovation is a particularly 
easy proof of a Khintchine-type inequality for matrices, patterned after [MJC + 14, Cor 7.3] and [Trol5b, 
Thm. 8.1], 

The ideas behind the proof of the lower bound in Theorem I are older. This estimate depends on 
generic considerations about the behavior of a sum of independent random variables in a Banach space. 
These techniques are explained in detail in [LT11, Ch. 6], Our presentation expands on a proof sketch 
that appears in the monograph [Trol5a, Secs. 5.1.2 and 6.1.2], 
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1.6. Target Audience. This paper is intended for students and researchers who want to develop a de¬ 
tailed understanding of the foundations of matrix concentration. The preparation required is modest. 

• Basic Convexity. Some simple ideas from convexity play a role, notably the concept of a convex 
function and Jensen’s inequality. 

• Intermediate Linear Algebra. The requirements from linear algebra are more substantial. The 
reader should be familiar with the spectral theorem for Hermitian (or symmetric) matrices, Rayleigh’s 
variational principle, the trace of a matrix, and the spectral norm. The paper includes reminders 
about this material. The paper elaborates on some less familiar ideas, including inequalities for 
the trace and the spectral norm. 

• Intermediate Probability. The paper demands some comfort with probability. The most impor¬ 
tant concepts are expectation and the elementary theory of conditional expectation. We develop 
the other key ideas, including the notion of symmetrization. 

Although many readers will find the background material unnecessary, it is hard to locate these ideas 
in one place and we prefer to make the paper self-contained. In any case, we provide detailed cross- 
references so that the reader may dive into the proofs of the main results without wading through the 
shallower part of the paper. 

1.7. Roadmap. Section 2 and Section 3 contain the background material from linear algebra and proba¬ 
bility. To prove the upper bound in Theorem I, the key step is to establish the result for the special case of 
a sum of fixed matrices, each modulated by a random sign. This result appears in Section 4. In Section 5, 
we exploit this result to obtain the upper bound in (1.5). In Section 6, we present the easier proof of the 
lower bound in (1.5). Finally, Section 7 shows that it is not possible to improve (1.5) substantially. 

2. Linear Algebra Background 

Our aim is to make this paper as accessible as possible. To that end, this section presents some back¬ 
ground material from linear algebra. Good references include [Hal74, Bha97, HJ13] . We also assume 
some familiarity with basic ideas from the theory of convexity, which may be found in the books [Lue69, 
Roc97, Bar02, BV04], 

2.1. Convexity. Let Lbea finite-dimensional linear space. A subset E a V is convex when 

x,yeE implies t-x + (1-t) -ycE for each r e [0,1], 

Let £ be a convex subset of a linear space V. A function /: E —• IR is convex if 

/(tjic+( l-r)y) <T-/(jc) + (l-r)-/(y) for all r £ [0,1] and all x,y e V. (2.1) 

We say that / is concave when -/ is convex. 

2.2. Vector Basics. Let C d be the complex linear space of d-dimensional complex vectors, equipped 
with the usual componentwise addition and scalar multiplication. The £2 norm ||-|| is defined on C d via 
the expression 

||jc || 2 := for each xeC d . 

The symbol * denotes the conjugate transpose of a vector. Recall that the £2 norm is a convex function. 

A family {ui,..., u d } c C d is called an orthonormal basis if it satisfies the relations 


The orthonormal basis also has the property 

E?= 

where l d is the d x d identity matrix. 


[0, i?j. 

UiU* =l d 
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2.3. Matrix Basics. A matrix is a rectangular array of complex numbers. Addition and multiplication 
by a complex scalar are defined componentwise, and we can multiply two matrices with compatible 
dimensions. We write M d] xd - for the complex linear space of d\ x d 2 matrices. The symbol * also refers 
to the conjugate transpose operation on matrices. 

A square matrix H is Hermitian when H = H*. Hermitian matrices are sometimes called conjugate 
symmetric. We introduce the set of dx d Hermitian matrices: 

H d :={HeM dxd :H= H*}. 

Note that the set Hrf is a linear space over the real field. 

An Hermitian matrix A e H f ; is positive semidefinite when 

u*Au>0 for each ueC d . 

It is convenient to use the notation A =<! H to mean that H - A is positive semidefinite. In particular, the 
relation 0 H is equivalent to H being positive semidefinite. Observe that 

0 =<: A and 0 ^ H implies 0 =4 a - (A + H) for each a > 0. 

In other words, addition and nonnegative scaling preserve the positive-semidefinite property. 

For every matrix B, both of its squares BB and B B are Hermitian and positive semidefinite. 

2.4. Basic Spectral Theory. Each Hermitian matrix He can be expressed in the form 

H= E?=i (2 - 2) 

where the A, are uniquely determined real numbers, called eigenvalues, and {«/} is an orthonormal basis 
for € d . The representation (2.2) is called an eigenvalue decomposition. 

An Hermitian matrix H is positive semidefinite if and only if its eigenvalues A, are all nonnegative. 
Indeed, using the eigenvalue decomposition (2.2), we see that 

U* Hu = ' u * 11 i u *i u = H/=l ' I u * u i I 2 - 

To verify the forward direction, select u = uj for each index j. The reverse direction should be obvious. 
We define a monomial function of an Hermitian matrix He by repeated multiplication: 

H° = l d , H 1 = H, H 2 = H H, H 3 = H ■ H 2 , etc. 

For each nonnegative integer r, it is not hard to check that 

H = £f =1 A i Ui u* implies H 1 = K u i u *i ■ ( 2 -3) 

In particular, H 2p is positive semidefinite for each nonnegative integer p. 

2.5. Rayleigh’s Variational Principle. The Rayleigh principle is an attractive expression for the maxi¬ 
mum eigenvalue A max (H) of an Hermitian matrix H e H^. This result states that 

A ma ix(-H) = max u* Hu. (2.4) 

IMI=i 

The maximum takes place over all unit-norm vectors u e C d . The identity (2.4) follows from the Lagrange 
multiplier theorem and the existence of the eigenvalue decomposition (2.2). Similarly, the minimum 
eigenvalue A m j n (H) satisfies 

A m in(H) — min u* Hu. (2.5) 

|| M|| = 1 

We can obtain (2.5) by applying (2.4) to -H. 

Rayleigh’s principle implies that order relations for positive-semidefinite matrices lead to order rela¬ 
tions for their eigenvalues. 

Fact 2.1 (Monotonicity). Let A, He be Hermitian matrices. Then 

A H implies Amax(-^l) — ^maxl H). 
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Proof. The condition A^. H implies that the eigenvalues of H A are nonnegative. Therefore, Rayleigh’s 
principle (2.5) yields 

0 < A mi n [H- A) = min [u* Hu- u* Au) < v*Hv- v* Av 
ll«ll=i 

for any unit-norm vector v. Select a unit-norm vector v for which A max (A) = v* Av, and then rearrange: 

^max (A) = v*Av< v*Hv< A max (H). 

The last relation is Rayleigh’s principle (2.4). □ 

2.6. The Trace. The trace of a square matrix B e M dxd is defined as 

trB: =Z?=i fo n- (2.6) 

It is clear that the trace is a linear functional on M rfxd . By direct calculation, one may verify that 

tr[BC] = tr[CB] for all B e M dxr and C e M rxd . 

This property is called the cyclicity of the trace. 

The trace of an Hermitian matrix H e Hrf can also be expressed in terms of its eigenvalues: 

trH= L?=i A i- (2-7) 

This formula follows when we introduce the eigenvalue decomposition (2.2) into (2.6). Then we invoke 
the linearity and the cyclicity properties of the trace, as well as the properties of an orthonormal basis. We 
also instate the convention that monomials bind before the trace: tr H r := tr[H r ] for each nonnegative 
integer r. 

2.7. The Spectral Norm. The spectral norm of a matrix B e M ll,xd2 is defined as 

IIBII := max \\Bu\\ . 

II «ll = 1 

The maximum takes place over unit-norm vectors u e C d ' 2 . We have the important identity 

||B|| 2 = ||B*B|| = \\BB* || for every matrix B. (2.8) 

Furthermore, the spectral norm is a convex function, and it satisfies the triangle inequality. 

For an Hermitian matrix, the spectral norm can be written in terms of the eigenvalues: 

||H|| = max{A max (Tf), -A m i n (H)} for each Hermitian matrix if. (2.9) 

As a consequence, 

|| ,A|| = AmaxfiA) for each positive-semidefinite matrix A. (2.10) 

This discussion implies that 

\\H\\ 2p = \\H 2p \\ for each Hermitian H and each nonnegative integer p. (2.11) 

Use the relations (2.3) and (2.9) to verify this fact. 

2.8. Some Spectral Norm Inequalities. We need some basic inequalities for the spectral norm. First, 
note that 

||A||<trA when A is positive semidefinite. (2.12) 

This point follows from (2.10) and (2.7) because the eigenvalues of a positive-semidefinite matrix are 
nonnegative. 

The next result uses the spectral norm to bound the trace of a product. 

Fact 2.2 (Bound for the Trace of a Product). Consider Hermitian matrices A,He H f /, and assume that A 
is positive semidefinite. Then 

tr[HA]< ||H||-trA 
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Proof. Introducing the eigenvalue decomposition A = A,- u i u* , we see that 

tr [HA] = Y_i Aitr[HuiU*] = X,- A, u* Hut < A max (H) • X,- A; < ||H||-trA. 

The first two relations follow from linearity and cyclicity of the trace. The first inequality depends on 
Rayleigh’s principle (2.4) and the nonnegativity of the eigenvalues AThe last bound follows from (2.9). 

□ 


We also need a bound for the norm of a sum of squared positive-semidefinite matrices. 


Fact 2.3 (Bound for a Sum of Squares). Consider positive-semidefinite matrices A\,...,A n e H f /. Then 

II X|Li II - max,-1 Ai II • || XfLt At [|. 

Proof. Let A be positive semidefinite. We claim that 


A 2 =4 M • A whenever A max (A) < M. 


(2.13) 


Indeed, introducing the eigenvalue decomposition A = AjUjU*, we find that 

M ■ A - A 2 = M ■ X . A,- Ui u* - X,- A? w, u* = X i ( M " A/)A f - m ; m* . 

The first relation uses (2.3). Since 0 < A/ < M, the scalar coefficients in the sum are nonnegative. There¬ 
fore, the matrix M ■ A - A 2 is positive semidefinite, which is what we needed to show. 

Select M := max,- A max C4,-). The inequality (2.13) ensures that 

A t =4 M-Ai for each index i. 

Summing these relations, we see that 

The monotonicity principle, Fact 2.1, yields the inequality 

'Wax (XI 1=1 ^i) — 'Wax [M- X,=l ^i) = M ■ A max (X.-i Aj'j . 

We have used the fact that the maximum eigenvalue of an Hermitian matrix is positive homogeneous. 
Finally, recall that, per (2.10), the spectral norm of a positive-semidefinite matrix equals its maximum 
eigenvalue. □ 


2.9. GM-AM Inequality for the Trace. We require another substantial matrix inequality, which is one (of 
several) matrix analogs of the inequality between the geometric mean and the arithmetic mean. 

Fact 2.4 (GM-AM Trace Inequality). Consider Hermitian matrices H,W,Y e H f /. For each nonnegative 
integer r and each integer q intherangeO< q<2r, 

tr [HW q HY 2r ~ q ) +tx[HW 2r ~ q HY q ]<tx[H 2 -[W 2r + Y 2r )}. (2.14) 

In particular, 

Y% =0 tr[HW c lHY 2r - c l] < ^XLLtr [ h 2 ■ [W 2r + Y 2r ]]. 

Proof. The result (2.14) is a matrix version of the following numerical inequality. For A,p > 0, 

A e p 1_s + A 1_0 p e < A + /i for each 6 e [0,1], (2.15) 

To verify this bound, we may assume that A, p > 0 because it is trivial to check when either A or p equals 
zero. Notice that the left-hand side of the bound is a convex function of 9 on the interval [0,1]. This point 
follows easily from the representation 

y-(g)._ x s p 1_e + A 1-0 // = e 01o 8 ;i+(1-0)lo s^ + e d-B)iogA+e\ogn 

The value of the convex function / on the interval [0,1] is controlled by the maximum value it achieves 
at one of the endpoints: 


fid) <max{/(0),/(l)} = A + p. 
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This inequality coincides with (2.15). 

To prove (2.14) from (2.15), we use eigenvalue decompositions. Thecase r = 0 is immediate, so we may 
assume that r > 1. Let q be an integer in the range 0 < q < 2r. Introduce eigenvalue decompositions: 

w = Lti^ u i u i and Y = Ly=i l J j v j v *j- 

Calculate that 

tr[HIV^HF 2 ^]=tr[H(j:f = iAfi* i ii;)H(j:J = i^ r -% J -i;;)^ 

= YA,j=lK> jZ j~ q ' tr i Hu ‘ U *i Hv ) v *j } {2 ' 16) 

The first identity relies on the formula (2.3) for the eigenvalue decomposition of a monomial. The second 
step depends on the linearity of the trace. In the last line, we rewrite the trace using cyclicity, and the 
inequality emerges when we apply absolute values. The representation | u* Hvj \ 2 emphasizes that this 
quantity is nonnegative, which we use to justify several inequalities. 

Invoking the inequality (2.16) twice, we arrive at the bound 

tr [HW q HY 2r ~ q ] + tr [HW 2r ~ q HY q ] < . [\Xi\ q |/xy \ 2r ~ q + Wi\ 2r ~ q |p/| ? ) • \u*Hvj | 2 

'J (2.17) 

*Y?i,i=A% r +rf)‘\ u *i Hv i\ • 

The second inequality is (2.15), with 6 = q/{2r) and A = A 2r and p = q 2r . 

1 J 

It remains to rewrite the right-hand side of (2.17) a more recognizable form. To that end, observe that 


tr [ HW q HY 2r ~ q ] + tr [HW 2r ~ q HY q ] 

'd r,2r , ,.2r 


^ Lij =i r + M/)' tr [Hut u* Hvj v* ] 

= tr [■»(L11 u i u i )■ H (Lti v i v *i )] + tr [■ H (Ltt u i u *) H ['Ll=if i f v j v l)] 


= tr JT Z 


■W 2r 1 + tr [ H 2 • 


^2 n 


In the first step, we return the squared magnitude to its representation as a trace. Then we use linearity 
to draw the sums back inside the trace. Next, invoke (2.3) to identify the powers W 2r and F° = I,/ and 
W° = I a and Y 2r . Last, use the cyclicity of the trace to combine the factors of H. The result (2.14) follows 
from the linearity of the trace. □ 


Remark 2.5 (The Power of Abstraction). There is a cleaner, but more abstract, proof of the inequal¬ 
ity (2.14). Consider the left- and right-multiplication operators 

\N :H~WH and Y : H>— HY. 


Observe that powers ofW and Y correspond to left- and right-multiplication by powers of W and F. Now, 
the operators W and Y commute, so there is a basis (orthonormal with respect to the trace inner product) 
for H a in which they are simultaneously diagonalizable. Representing the operators in this basis, we can 
use (2.15) to check that 

yyqy2r-q + yy2r-qyq ^ yy2r + y2r 

Now, calculate that 

tr [HW q HY 2r ~ q ] +tr [HW 2r ~ q HY q ] = tr [H (W' ? Y 2r “' ? + W 2r “' ? Y‘ 7 )(H)] 

<tv[H-(\N 2r + Y 2r )(H)] 

= tr [H 2 -[W 2r + Y 2 ’)]. 


We omit the details. 
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2.10. The Hermitian Dilation. Last, we introduce the Hermitian dilation of a rectangular matrix 

Be M dlXdz . This is the Hermitian matrix 


JT(B) := 


0 B 
B* 0 


G H rfl+d2 . 


Note that the map is real linear. By direct calculation, 


jT(B ) 2 = 


BB* 0 
0 B*B 


We also have the spectral-norm identity 


\\3t?{B)\\ = ||B||. 


(2.18) 


(2.19) 


( 2 . 20 ) 


To verify (2.20), calculate that 


\\J^{B)\\ 2 = ||jT(B) 2 


BB* 

0 


0 

B*B 


= max{ ||BB* ||, ||B*B|| } 


IBII 


The first identity is (2.11); the second is (2.19). The norm of a block-diagonal Hermitian matrix is the 
maximum spectral norm of a block, which follows from the Rayleigh principle (2.4) with a bit of work. 
Finally, invoke the property (2.8). 


3. Probability Background 

This section contains some background material from the field of probability. Good references include 
the books [LT11, GS01, Taol2]. 

3.1. Expectation. The symbol E denotes the expectation operator. We will not define expectation for¬ 
mally or spend any energy on technical details. No issues arise if we assume, for example, that all random 
variables are bounded. 

We use brackets to enclose the argument of the expectation when it is important for clarity, and we 
instate the convention that nonlinear functions bind before expectation. For instance, EX P := E[X P ] and 
Emax; Xj := E[max,- X;]. 

Sometimes, we add a subscript to indicate a partial expectation. For example, if / is a random variable, 
Ej refers to the average over /, with all other random variables fixed. We only use this notation when / 
is independent from the other random variables, so there are no complications. In particular, we can 
compute iterated expectations: E[E/[-]] = E[-] whenever all the expectations are finite. 

3.2. Random Matrices. A random matrix is a matrix whose entries are complex random variables, not 
necessarily independent. We compute the expectation of a random matrix Z componentwise: 

(E [Z])ij = E [Zij] for each pair (i, j) of indices. 

As in the scalar case, if W and Z are independent, 

E [WZ\ = (EW)(EZ). 

Since the expectation is linear, it also commutes with all of the simple linear operations we perform on 
matrices. 

It suffices to take a naive view of independence, expectation, and so forth. For the technically inclined, 
let (Q, P) be a probability space. A d\ x d 2 random matrix Z is simply a measurable function 

Z :Q.->M dlxd2 . 

A family {Z,: i = 1,..., n} of random matrices is independent when 

P {Zi e Ei for i = 1,..., n} = ]~[" =1 P {Z, : e Ed. 
for any collection of Borel 1 subsets c M d] xd2 . 


^Open sets in M dl xd2 are defined with respect to the metric topology induced by the spectral norm. 
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3.3. Inequalities for Expectation. We need several basic inequalities for expectation. We set these out 
for future reference. Let X, Y be (arbitrary) real random variables. The Cauchy-Schwarz inequality states 
that 

|E[XF]| < (EX 2 ) 1/2 -(EF 2 ) 1/2 . (3.1) 

For r > 1, the triangle inequality states that 

(Eix+yr) 1 /r <(Eixr) 1 /r +(EiFr) 1/r . (3.2) 

Each of these inequalities is vacuous precisely when its right-hand side is infinite. 

Jensen’s inequality describes how expectation interacts with a convex or concave function; cf. (2.1). 
Let X be a random variable taking values in a finite-dimensional linear space V, and let / : V —► R be a 
function. Then 

/(EX)<E/(X) when / is convex, and 
E/(X)</(EX) when / is concave. 

The inequalities (3.3) also hold when we replace E with a partial expectation. Let us emphasize that these 
bounds do require that all of the expectations exist. 


3.4. Symmetrization. Symmetrization is an important technique for studying the expectation of a func¬ 
tion of independent random variables. The idea is to inject auxiliary randomness into the function. Then 
we condition on the original random variables and average with respect to the extra randomness. When 
the auxiliary random variables are more pliable, this approach can lead to significant simplifications. 

A Rademacher random variable e takes the two values +1 with equal probability. The following result 
shows how we can use Rademacher random variables to study a sum of independent random matrices. 


Fact 3.1 (Symmetrization). LetSi,...,S n e M dixdz be independent random matrices. Let E\,...,e n be in¬ 
dependent Rademacher random variables that are also independent from the random matrices. For each 


r> 1, 


|-(Eiix:- 7 = 1 ^^dr) 1/r ^ (Eiix-irC^-Es.-Jir) 17 '' <2-(E|ix:- 7 = 1 ^tSt 


Hr 


This result holds whenever E || S,-1|' < oo for each index i. 


Proof. For notational simplicity, assume that r = 1. We discuss the general case at the end of the argu¬ 
ment. 

Let {S'.: i = 1,...,«} be an independent copy of the sequence {S;: i = 1,..., n}, and let E' denote partial 
expectation with respect to the independent copy. Then 

E 1 E"=i (Si - E Si) II =E||E"=i [(Si - E Si) - E '(S' - ESi)] I 

< E [E' ||E” =1 [(Si - ES,0 - (S( - E Sf] | ] 

= E E=t^-s;)|. 

The first identity holds because E'S'- = ES; by identical distribution. Since the spectral norm is convex, 
we can apply Jensen’s inequality (3.3) conditionally to draw out the partial expectation E'. Last, we com¬ 
bine the iterated expectation into a single expectation. 

Observe that S; - S', has the same distribution as its negation S'. - S,-. It follows that the independent 
sequence {e,-(Sj - S'.) : i = 1,...,«} has the same distribution as {S, - S'. : i = 1,...,«}. Therefore, the 
expectation of any nonnegative function takes the same value for both sequences. In particular, 

E IIE"=1 (S *' - S 'j II = E IIE"=t £ *(S; - S' t ) || 

< e iie" = io-s ; | + e »e- = i(-^ s :ii 

= 2 e IE” = i^S/||. 

The second step is the triangle inequality, and the last line follows from the identical distribution of {e,-S,-} 
and {-£(S'.J. Combine the last two displays to obtain the upper bound. 
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To obtain results for r > 1, we pursue the same approach. We require the additional observation that 
|| • ||' is a convex function, and we also need to invoke the triangle inequality (3.2). Finally, we remark that 
the lower bound follows from a similar procedure, so we omit the demonstration. □ 


4. The Expected Norm of a Matrix Rademacher Series 

To prove Theorem I, our overall strategy is to use symmetrization. This approach allows us to reduce 
the study of an independent sum of random matrices to the study of a sum of fixed matrices modu¬ 
lated by independent Rademacher random variables. This type of random matrix is called a matrix 
Rademacher series. In this section, we establish a bound on the spectral norm of a matrix Rademacher 
series. This is the key technical step in the proof of Theorem I. 

Theorem 4.1 (Matrix Rademacher Series). Let H\,...,H n be fixed Hermitian matrices with dimension d. 
Lete\,...,e n be independent Rademacher random variables. Then 

( E II£"= i £ iHi II 2 ) 1/2 - • II£” = 1 Hf | 1/2 . (4.1) 

The proof of Theorem 4.1 occupies the bulk of this section, beginning with Section 4.2. The argument 
is really just a fancy version of the familiar calculation of the moments of a centered standard normal 
random variable; see Section 4.8 for details. 


4.1. Discussion. Before we establish Theorem 4.1, let us make a few comments. First, it is helpful to in¬ 
terpret the result in the same language we have used to state Theorem I. Introduce the matrix Rademacher 
series 




n 

i =1 


Si Hi. 


Compute the matrix variance, defined in (1.2): 


viX) := \\EX Z \\ = 


=||£- =1 Hf||. 


We may rewrite Theorem 4.1 as the statement that 


(E||X|| 2 ) 1/2 < yj (1 + 2riogdl) • v{X). 

In other words, Theorem 4.1 is a sharper version of Theorem I for the special case of a matrix Rademacher 
series. 

Next, we have focused on bounding the second moment of ||X|| because this is the most natural form 
of the result. Note that we also control the first moment because of Jensen’s inequality (3.3): 


E ll£Li^^dN( E ll£r=i £ ^4l 2 ) 1/2 ^^i+2[iogdi.||£^ 1 H ; 2 


r 2|| 1/2 


(4.2) 


A simple variant on the proof of Theorem 4.1 provides bounds for higher moments. 

Third, the dimensional factor on the right-hand side of (4.1) is asymptotically sharp. Indeed, let us 
write Kid) for the minimum possible constant in the inequality 

( E ||£" =1 a Hi || 2 ) 1/2 < Kid ) • ||£" =1 H] 1 1/2 for Hi e H d and n e N. 

The example in Section 7.1 shows that 

Kid) > \J 2 \ 0 gd. 

In other words, (4.1) cannot be improved without making further assumptions. 

Theorem 4.1 is a variant on the noncommutative Khintchine inequality, first established by Lust- 
Piquard [LP86] and later improved by Pisier [Pis98] and by Buchholz [BucOl]. The noncommutative 
Khintchine inequality gives bounds for the Schatten norm of a matrix Rademacher series, rather than for 
the spectral norm. Rudelson [Rud99] pointed out that the noncommutative Khintchine inequality also 
implies bounds for the spectral norm of a matrix Rademacher series. In our presentation, we choose to 
control the spectral norm directly. 
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4.2. The Spectral Norm and the Trace Moments. To begin the proof of Theorem 4.1, we introduce the 
random Hermitian matrix 

x -=L'L £ i H i (4.3) 

Our goal is to bound the expected spectral norm of X. We may proceed by estimating the expected trace 
of a power of the random matrix, which is known as a trace moment. Fix a nonnegative integer p. Observe 
that 

(E||X|| 2 ) 1/2 < (E||X|| 2p ) 1/(2p) = (E||X 2p ||) 1/(2p) < (EtrX 2p ) 1/(2p) . (4.4) 

The first identity is Jensen’s inequality (3.3), applied to the concave function t —► t llp . The second relation 
is (2.11). The final inequality is the bound (2.12) on the norm of the positive-semidefinite matrix X 2p by 
its trace. 

Remark 4.2 (Higher Moments). It should be clear that we can also bound expected powers of the spectral 
norm using the same technique. For simplicity, we omit this development. 

4.3. Summation by Parts. To study the trace moments of the random matrix X, we rely on a discrete 
analog of integration by parts. This approach is clearer if we introduce some more notation. For each 
index i, define the random matrices 

X +i :=H i + Y.j* i £jHj and JL f := -H f + Ej^jHj 

In other words, the distribution of X Ej i is the conditional distribution of the random matrix X given 
the value £,• of the z'th Rademacher variable. This interpretation depends on the assumption that the 
Rademacher variables are independent. 

Beginning with the trace moment, observe that 


EtrX 2p = Etr [X ■ X 2p_1 ] 

= E"=iE[E tl tr[ E/ H,.jr 2 o- 1 ]] 


4 EU «■■[«.•'(*5 


2p—1 j^2p-l 


(4.5) 


In the second step, we simply write out the definition (4.3) of the random matrix X and use the linearity 
of the trace to draw out the sum. Then we write the expectation as an iterated expectation. To reach the 
next line, write out the partial expectation using the notation X+i and the linearity of the trace. 


4.4. A Difference of Powers. Next, let us apply an algebraic identity to reduce the difference of powers 
in (4.5). For matrices W, Y e H^, it holds that 

W 2p ~ 1 - Y 2p ~ l = W q (W-Y)Y 2p ~ 2 ~ cl . (4.6) 


To check this expression, just expand the matrix products and notice that the sum telescopes. 
Introducing the relation (4.6) with W = X+j and Y = X-j into the formula (4.5), we find that 


ElrX 2p = IEtr [Hi ■ EE-*-i) 


2p-2-q 


= E.,I 2 ,5 2 Etr 


fj.x 4 TJ-X 2p 2 ^ 

Z Mil —i 


(4.7) 


Linearity of the trace allows us to draw out the sum over q, and we have used the observation that X + , - 
X-i = 2 H{. 
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4.5. A Bound for the Trace Moments. We are now in a position to obtain a bound for the trace moments 
of X. Beginning with (4.7), we compute that 


EtrX^ELE;,:, Etr 
<E” , — Etr 


H ■ H-X 2p ~ 2 ~^ 


Hf-( 


x 2p-2 + x 2p-2 


(4.8) 


= (2p-l)-L- =1 Etr[Hf.(E ei X 2 P- 2 )] 

= (2p - 1) • Etr [Q^” =1 H 2 ) -X 2p_2 ] 

< (2p- 1) • ||£" =1 H i\\ -EtrX 2p “ 2 . 

The bound in the second line is the trace GM-AM inequality, Fact 2.4, with r = p- 1 and W = X +; - and 
Y = X-i. To reach the third line, observe that the parenthesis in the second line is twice the partial 
expectation of X 2p ~ 2 with respect to £,-. Afterward, we use linearity of the expectation and the trace to 
draw in the sum over i, and then we combine the expectations. Last, invoke the trace inequality from 
Fact 2.2. 

4.6. Iteration and the Spectral Norm Bound. The expression (4.8) shows that the trace moment is con¬ 
trolled by a trace moment with a smaller power: 

EtrX 2p < [2p — 1) • [|£" =1 H 2 \\-EtrX 2p ~ 2 . 

Iterating this bound p times, we arrive at the result 

EtrX 2p < (2p-l)!!- IIV" H 2 \\ P -tvX° 

(4-9) 

= d-{2p-W-\\Y,”=i H U ■ 

The double factorial is defined as (2p - 1)!! := (2p - l)(2p -3)(2p -5) ••• (5) (3) (1). 

The expression (4.4) shows that we can control the expected spectral norm of X by means of a trace 
moment. Therefore, for any nonnegative integer p, it holds that 

E||X|| < (EtrX 2p ) 1/(2p) <(d.(2p-l)!!) 1/t2p) .||^” =1 i/ 2 || 1/2 . (4.10) 

The second inequality is simply our bound (4.9). All that remains is to choose the value of p to minimize 
the factor on the right-hand side. 

4.7. Calculating the Constant. Finally, let us develop an accurate bound for the leading factor on the 
right-hand side of (4.10). We claim that 

t2p+l\ p 


(2p-l)!!< 

Given this estimate, select p = [log d] to reach 

[d- (2p- l)!!) 1/t2p) < d 1/(2p) ^ 


2p+ 1 


< V / 2p+l = y 1 + 2 flog dl 


(4.11) 


(4.12) 


Introduce the inequality (4.12) into (4.10) to complete the proof of Theorem 4.1. 
To check that (4.11) is valid, we use some tools from integral calculus: 

l°g((2p~ 1)!!) = lo g(2z’ + !) 


= ^ log(2 • 0 + 1) + Xf=i 1 l°g( 2 *’ + D + ^ log(2p + 1) 


— 2 lo g(2p + !) 


< f log(2x 
Jo 

= plog(2p+l)-p 


+ l)dx- ^log(2p+ 1) 
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The bracket in the second line is the trapezoid rule approximation of the integral in the third line. Since 


the integrand is concave, the trapezoid rule underestimates the integral. Exponentiating this formula, 
we arrive at (4.11). 

4.8. Context. The proof of Theorem 4.1 is really just a discrete, matrix version of the familiar calculation 


of the (2p)th moment of a centered normal random variable. Let us elaborate. Recall the Gaussian 
integration by parts formula: 


E[y-/(y)l = cr 2 • E[/'(y)] 


(4.13) 


where y ~ normal(0,u 2 ) and / : R — K is any function for which the integrals are finite. This result 


follows when we write the expectations as integrals with respect to the normal density tand invoke the 


usual integration by parts rule. Now, suppose that we wish to compute the (2p)th moment of y. We have 


Ey 2p = E [y • y 2p_1 ] = (2p - 1) • a 2 ■ Ey 2p “ 2 . 

The second identity is just (4.13) with the choice fit) = t 2p ~ l . Iterating (4.14), we discover that 

Ey 2p = [2p — 1)!! • a 2p . 

In Theorem 4.1, the matrix variance parameter v(X) plays the role of the scalar variance a 2 . 


(4.14) 


In fact, the link with Gaussian integration by parts is even stronger. Consider a matrix Gaussian series 



where {y, } is an independent family of standard normal variables. If we replace the discrete integration 
by parts in the proof of Theorem 4.1 with Gaussian integration by parts, the argument leads to the bound 



This approach requires matrix calculus, but it is slightly simpler than the argument for matrix Rademacher 
series in other respects. See [Trol5b, Thm. 8.1] for a proof of the noncommutative Khintchine inequality 
for Gaussian series along these lines. 


5. Upper Bounds for the Expected Norm 


We are now prepared to establish the upper bound for an arbitrary sum of independent random ma¬ 
trices. The argument is based on the specialized result, Theorem 4.1, for matrix Rademacher series. It 
proceeds by steps through more and more general classes of random matrices: first positive semidefinite, 
then Hermitian, and finally rectangular. Here is what we will show. 

Theorem 5.1 (Expected Norm: Upper Bounds). Define the dimensional constant C[d) := 4(1+ 2[logdl). 
The expected spectral norm of a sum of independent random matrices satisfies the following upper bounds. 

(1) The Positive-Semidefinite Case. Consider an independent family {T\,..., T n ] of random d x d 
positive-semidefinite matrices, and define the sum 



Then 


E\\W\\ < ||E W|| 1/2 + VC(d) ■ (Ernax,-1| T t ||) 


(5.1) 


(2) The Centered Hermitian Case. Consider an independent family {Yj,.. .,Y n } of random d x d Her¬ 
mitian matrices with E F, = 0 for each index i, and define the sum 



(E ||X|| 2 ) 1/2 < y/cufi • II EX 2 1 1/2 + C(d) • (Emax,: || Y t || 2 ) 1/2 . 


Then 


( 5 . 2 ) 
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(3) The Centered Rectangular Case. Consider an independent family {Si,..., S n } of random d\ x c/ 2 
matrices withESj = 0 for each index i, and define the sum 

z --= rus . 

Then 

E \\Z\\ < y/Cid)- max) ||E [ZZ*] || 1/2 , ||E [Z*Z] || 1/2 } + C{d) • (Emax; ||S,-1| 2 ) 1/2 (5.3) 

whered := d\ + 

The proof of Theorem 5.f takes up the rest of this section. The presentation includes notes about the 
provenance of various parts of the argument. 

The upper bound in Theorem I follows instantly from Case (3) of Theorem 5.1. We just introduce the 
notation v(Z) for the variance parameter, and we calculate that 

E l^*] =LlASiS*]. 

The first expression follows immediately from the definition of Z and the linearity of the expectation; 
the second identity holds because the random matrices S; are independent and have mean zero. The 
formula for E [Z*Z] is valid for precisely the same reasons. 


5.1. Proof of the Positive-Semidefinite Case. Recall that W is a random d x d positive-semidefinite ma¬ 
trix of the form 

W: =L'L Ti where the Tj are positive semidefinite. 

Let us introduce notation for the quantity of interest: 

E := E|| W|| = E||^” =1 Tf || 


By the triangle inequality for the spectral norm, 

£ ^IIIL Er /ll +E IIILi^- Er /)INIII"=i E ^ll +2E llir=i^^ 


The second inequality follows from symmetrization, Fact 3.1. In this expression, {c,} is an independent 
family of Rademacher random variables, independent from {Tj}. Conditioning on the choice of the ran¬ 
dom matrices Tj, we apply Theorem 4.1 via the bound (4.2): 


E ILL £ i T i II = E l E * »E” =1 £ i T i\\]^\f l+2 nog • E [ ILL Tf \\ 1/2 

The operator E e averages over the choice of the Rademacher random variables, with the matrices Tj fixed. 
Now, since the matrices Tj are positive-semidefinite, 


E 


I Y " T 2 


,2 II1/2 


< E 


(max, || Tj 


\ 1/2 


W=i T i 


11/2 


<(Emax;||r / ||) 1/2 -(E||^ =i r;||) 1/2 
= (Emax; || Tj\\ ) 1/2 -E 112 . 


The first inequality is Fact 2.3, and the second is the Cauchy-Schwarz inequality (3.1) for expectation. In 
the last step, we identified a copy of the quantity E. 

Combine the last three displays to see that 


E < | Yti E T i II + V 14EL + 2 r lo § ) • ( 1 Emax,-1| Tj \ 


\ 1/2 77I/2 


For any a, f > 0, the quadratic inequality t 2 < a + fit implies that 


t < 


1 


f>+ J f z + 4a 


— 2 iP + P + 2 v / ®] - \/® + f 


(5.4) 
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because the square root is subadditive. Applying this fact to the quadratic relation (5.4) for E 112 , we 
obtain 

E 1 ' 2 ^ IIEL E II 1/2 + ^4(1+ 2 [log dl) • (Ernax; || T f || ) 1/2 . 

Square both sides to reach the conclusion (5.1). 

This argument is adapted from Rudelson’s paper [Rud99], which develops a version of this result for 
the case where the matrices T, have rank one; see also [RV07]. The paper [Tro08] contains the first esti¬ 
mates for the constants. Magen & Zouzias [MZ11] observed that similar considerations apply when the 
matrices 77 have higher rank. The complete result (5.1) first appeared in [CGT12, App.]. The constants 
in this paper are marginally better. Related bounds for Schatten norms appear in [MJC + 14, Sec. 7] and 
in [JZ13] . 

The results described in the last paragraph are all matrix versions of the classical inequalities due to 
Rosenthal [Ros70b, Lem. 1], These bounds can be interpreted as polynomial moment versions of the 
Chernoff inequality. 


5.2. Proof of the Hermitian Case. The result (5.2) for Hermitian matrices is a corollary of Theorem 4.1 
and the positive-semidefinite result (5.1). Recall that X is a d x d random Hermitian matrix of the form 


X := YJl =l Yi where E Y t = 0. 


We may calculate that 




2H/2 


<2 E 




1/2 


< V / 4(l + 2riogdl). E 1X^17 


1/2 


The first inequality follows from the symmetrization procedure, Fact 3.1. The second inequality applies 
Theorem 4.1, conditional on the choice of Yj. The remaining expectation contains a sum of independent 
positive-semidefinite matrices. Therefore, we may invoke (5.1) with 77 = Y 2 . We obtain 


E||V" . if II < || y n ,E Yj 

II l II \\^l=l l 


2II1/2 


+ 


4(1 + 2 riog d \) • (Emax, || Y? ||) 


1/2 


Combine the last two displays to reach 


fE||X || 2 ) 1/2 < \[a 




Rewrite this expression to reach (5.2). 

A version of the result (5.2) first appeared in [CGT12, App.]; the constants here are marginally better. 
Related results for the Schatten norm appear in the papers [JX03, JX08, MJC + 14, JZ13] . These bounds 
are matrix extensions of the scalar inequalities due to Rosenthal [Ros70b, Thm. 3] and to Rosen [Ros70a, 
Thm. 1]; see also Nagaev-Pinelis [NP77, Thm. 2]. They can be interpreted as the polynomial moment 
inequalities that sharpen the Bernstein inequality. 


5.3. Proof of the Rectangular Case. Finally, we establish the rectangular result (5.3). Recall that Z is a 
cl] x d-i random rectangular matrix of the form 

Z := Y^[L] Si where ES, =0. 

Set d:= d,\ + dz, and form a random d x d Hermitian matrix X by dilating Z: 

X:=^(Z) = £" =1 Jr(Si). 

The Hermitian dilation is defined in (2.18); the second relation holds because the dilation is a real- 
linear map. 
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Evidently, the random matrix X is a sum of independent, centered, random Hermitian matrices . 

Therefore, we may apply (5.2) to X to see that 

(E \\J4?{Z)\\ 2 ) 112 < yj 4(1 + 2 [log rf]) • ||E [ (Z) 2 ] || 1/2 + 4(1 + 2[logdl) • (Emax* \\j4?{Si)\\ 2 ) 112 . (5.5) 

Since the dilation preserves norms (2.20), the left-hand side of (5.5) is exactly what we want: 

(E||^(Z)|| 2 ) 1/2 = (E||Z|| 2 ) 1/2 . 


To simplify the first term on the right-hand side of (5.5), invoke the formula (2.19) for the square of the 
dilation: 


e[jT(Z) 2 ]|| 


E[ZZ*] 0 
0 E[Z*Z] 


= max{||E[ZZ*]||,||E[Z*Z]||}. 


The second identity relies on the fact that the norm of a block-diagonal matrix is the maximum norm of 
a diagonal block. To simplify the second term on the right-hand side of (5.5), we use (2.20) again: 

ll^(Si)ll = IIS(||. 


Introduce the last three displays into (5.5) to arrive at the result (5.3) . 

The result (5.3) first appeared in the monograph [Trol5a, Eqn. (6.16)] with (possibly) incorrect con¬ 
stants. The current paper contains the first complete presentation of the bound. 


6. Lower Bounds for the Expected Norm 


Finally, let us demonstrate that each of the upper bounds in Theorem 5.1 is sharp up to the dimen¬ 
sional constant C(d). The following result gives matching lower bounds in each of the three cases. 


Theorem 6.1 (Expected Norm: Lower Bounds). The expected spectral norm of a sum of independent ran¬ 
dom matrices satisfies the following lower bounds. 

(1) The Positive-Semidefinite Case. Consider an independent family {Ti,..., T n ] of random d x d 
positive-semidefinite matrices, and define the sum 

W: =LU T i 

Then 


E\\W\\ > - 

4 


||EW|| 1/2 + (Emax; ||I7H) 


1/2 


( 6 . 1 ) 


(2) The Centered Hermitian Case. Consider an independent family {Yi,..., Y n ] of random dx d Her¬ 
mitian matrices with E F, = 0 for each index i, and define the sum 


Then 


X:=E“.,T. 


(EII XlC) 1 ' 2 £ i I E Jirqi 1/2 + i (E max, IIK, II 2 ) 1/2 . 


( 6 . 2 ) 


(3) The Centered Rectangular Case. Consider an independent family {Si,..., S„} of random d\ x d 2 
matrices withESj = 0 for each index i, and define the sum 


z^Us, 


Then 


E||Z|| > -max{|[E|ZZ 


11/2 


EZ’Z 


, 1/21 


j + - (Emax,- ||S;|| 2 ) 


211/2 


(6.3) 


The rest of the section describes the proof of Theorem 6.1. 

The lower bound in Theorem I is an immediate consequence of Case (3) of Theorem 6.1. We simply 
introduce the notation v(Z) for the variance parameter. 
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6.1. The Positive-Semidefinite Case. The lower bound (6.1) in the positive-semidefinite case is rela¬ 
tively easy. Recall that 

W -=L’L Ti where the Tj are positive semidefinite. 

First, by Jensen’s inequality (3.3) and the convexity of the spectral norm, 

E|| W|| > IIEWII. (6.4) 

Second, let / be the minimum value of the index i where max,-1| Tj || is achieved; note that I is a random 
variable. Since the summands T, are positive semidefinite, it is easy to see that 

Ti<L'U T i- 

Therefore, by the norm identity (2.10) for a positive-semidefinite matrix and the monotonicity of the 
maximum eigenvalue, Fact 2.1, we have 

max; || Tj. || = || 7j|| = T max (T/) < A max ,_i 7 1 ;) = ||Xl;=i ^ II = II • 

Take the expectation to arrive at 

Emax; || r,-|| < E||W||. (6.5) 

Average the two bounds (6.4) and (6.5) to obtain 

1 r 

E||IV|| >-[ ||EW||+Emax£ || I;||]. 

To reach (6.1), apply the numerical fact that 2 {a + b) > [y/a + Vb) 2 , valid for all a,b> 0. 

6.2. Hermitian Case. The Hermitian case (6.2) is similar in spirit, but the details are a little more in¬ 
volved. Recall that 

X:=ru Y ‘ whereEY > = 0 

First, using the identity (2.11), we have 

(E||X|| 2 ) 1/2 = (E||X 2 ||) 1/2 > ||EX 2 || 1/2 . (6.6) 

The second relation is Jensen’s inequality (3.3). 

To obtain the other part of our lower bound, we use the lower bound from the symmetrization result, 
Fact 3.1: 

EIIXII 2 = t |E" =1 riII 2 2 JE|E" =1 C,Y ,f 

where {c;} is an independent family of Rademacher random variables, independent from {Y,}. Now, we 
condition on the choice of {Y;}, and we compute the partial expectation with respect to the e;. Let I 
be the minimum value of the index i where max; || Y; || 2 is achieved. By Jensen’s inequality (3.3), applied 
conditionally, 

E e ||E"=l £ i Y i II 2 ^ ll E E"=l £ i Y i I £ l] f = E r/ UlYrf 2 = max; || Y; || 2 . 

Combining the last two displays and taking a square root, we discover that 

(E||X|| 2 ) 1/2 > i(Emax; || Y;|| 2 ) 1/2 . (6.7) 

Average the two bounds (6.6) and (6.7) to conclude that (6.2) is valid. 
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6.3. The Rectangular Case. The rectangular case (6.3) follows instantly from the Hermitian case when 
we apply (6.2) to the Hermitian dilation. Recall that 

Z'■='Y^i=\Si where E Sj = 0. 

Define a random matrix X by applying the Hermitian dilation (2.18) to Z: 

X:= Jf(Z) = £" =1 Jr(S,0. 

Since X is a sum of independent, centered, random Hermitian matrices, the bound (6.2) yields 
(Ell JT(Z)|| 2 ) 1/2 > i ||e[JW) 2 ]|| + ^(EmaXi ||^(S,-)|| 2 ) 1/2 . 

Repeating the calculations in Section 5.3, we arrive at the advertised result (6.3). 


7. Optimality of Theorem I 

The lower bounds and upper bounds in Theorem I match, except for the dimensional factor C(d). In 
this section, we show by example that neither the lower bounds nor the upper bounds can be sharpened 
substantially. More precisely, the logarithms cannot appear in the lower bound, and they must appear 
in the upper bound. As a consequence, unless we make further assumptions, Theorem I cannot be im¬ 
proved except by constant factors and, in one place, by an iterated logarithm. 


7.1. Upper Bound: Variance Term. First, let us show that the variance term in the upper bound in (1.5) 
must contain a logarithm. This example is drawn from [Trol5a, Sec. 6.1.2]. 

For a large parameter n, consider the d x d random matrix 


As before, {£,y} is an independent family of Rademacher random variables, and E a is a dx d matrix with 
a one in the (/', i) position and zeroes elsewhere. The variance parameter satisfies 


v[Z) = 


y d y n If- 

2-i=iZ-/=i 


= n id n = i. 


The large deviation parameter satisfies 


L 2 = Emax,-j 


£ i / E j j 

yn 1 


i 

n 


Therefore, the variance term drives the upper bound (1.5). For this example, it is easy to estimate the 
norm directly. Indeed, 


E ||Z|| 2 




= E max |y;| 2 
i=l, ...,d 


2 log d. 


Here, {yd is an independent family of standard normal variables, and the first approximation follows 
from the central limit theorem. The norm of a diagonal matrix is the maximum absolute value of one 
of the diagonal entries. Last, we use the well-known fact that the expected maximum among d squared 
standard normal variables is asymptotic to 2 log <7. In summary, 


(E||Z|| 2 ) 1/2 ~ y/2\ogd-v{X). 

We conclude that the variance term in the upper bound must carry a logarithm. Furthermore, it follows 
that Theorem 4.1 is numerically sharp. 
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7.2. Upper Bound: Large-Deviation Term. Next, we verify that the large-deviation term in the upper 
bound in (1.5) must also contain a logarithm, although the bound is slightly suboptimal. This example 
is drawn from [Trol5a, Sec. 6.1.2]. 

For a large parameter n, consider the d x d random matrix 


Z--=EiXU [Stj-n-^Eu 

where {5,-j} is an independent family of Bernoulli (n -1 ) random variables. That is, Sij takes only the 
values zero and one, and its expectation is n~ l . The variance parameter for the random matrix is 


v{Z) = 


Lti LL^u-H-y-Eu = ELELh-^i-O-e 


The large deviation parameter is 

L 2 = Emaxij ||(d/y - n _1 ) • E fl - [| 2 ~ 1. 


Therefore, the large-deviation term drives the upper bound in (1.5): 


(EII Z|| 2 ) 1/2 < yj 4(1 + 2 riog d ]) + 4(1 + 2 riog d \). 
On the other hand, by direct calculation 


(E||Z|| 2 ) 1/2 



__, A 

2’ 

1/2 

E 

E? =1 (Q/-D-En 




E max IQ;- 1| 


\ 1/2 


const • 


logd 
log log d' 


Here, {Q, | is an independent family of poisson(I) random variables, and the first approximation follows 
from the Poisson limit of a binomial. The second approximation depends on a (messy) calculation for the 
expected squared maximum of a family of independent Poisson variables. We see that the large deviation 
term in the upper bound (1.5) cannot be improved, except by an iterated logarithm factor. 


7.3. Lower Bound: Variance Term. Next, we argue that there are examples where the variance term in 
the lower bound from (1.5) cannot have a logarithmic factor. 

Consider a d x d random matrix of the form 

z ' = Ei,j=i £ Q e Q- 

Here, {£/y} is an independent family of Rademacher random variables. The variance parameter satisfies 


v{Z) = max{|E/j=t ( E£ q) • e Q e q|| - ||Efj=i ( Ee fj) -E^-Eq||} = max{||d-] 


\d-\A} = d. 


The large-deviation parameter is 

L 2 = Emax/j- ||c/yE;y| 2 = 1. 

Therefore, the variance term controls the lower bound in (1.5): 


(E ||Z|| 2 ) 1/2 > \fcd + c. 


Meanwhile, it can be shown that the norm of the random matrix Z satisfies 

(E||Z|| 2 ) 1/2 « slid. 


See the paper [BH14] for an elegant proof of this nontrivial result. We see that the variance term in the 
lower bound in (1.5) cannot have a logarithmic factor. 
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7.4. Lower Bound: Large-Deviation Term. Finally, we produce an example where the large-deviation 
term in the lower bound from (1.5) cannot have a logarithmic factor. 

Consider a dx d random matrix of the form 

Z : = E?=i P i E a- 

Here, {Pfi is an independent family of symmetric random variables whose tails satisfy 


4 


P{\Pi\>t} = 


1 , 


t> 1 
t< 1. 


The key properties of these variables are that 

EP 2 =2 and E max P 2 a const- d 2 . 

1 i=l. d 1 

The second expression just describes the asymptotic order of the expected maximum. We quickly com¬ 
pute that the variance term satisfies 

vw = \\Ltii EP t) E 4 =2 - 

Meanwhile, the large-deviation factor satisfies 

L 2 = E max ||P/E,:,;|| 2 = E max |P;| 2 = 
i=l,...,d i=l,...,d 

Therefore, the large-deviation term drives the lower bound (1.5): 

n2r 1/2 


■ const - d. 


(E||Z|| 2 ) > const -d. 


On the other hand, by direct calculation, 


(E||Z|| 2 ) 1/2 = E £" =1 P,-E 


1/2 


E max |P,-| 


1/2 


const -d. 


We conclude that the large-deviation term in the lower bound (1.5) cannot carry a logarithmic factor. 
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